Efficient and Numerically Stable Sparse Learning.

Cross Validation Framework to Choose amongst Models and Datasets for Transfer Learning.

The next generation of transportation systems
Using immersive video to evaluate future
Cumulonimbus forecasting based on rough set and artificial immune algorithm.

The next generation of transportation systems
Using immersive video to evaluate future
On community outliers and their efficient detection in information networks.

Generalized and Heuristic%2DFree Feature Construction for Improved Accuracy.

Multi%2Dlabel Classification without the Multi%2Dlabel Cost.

Predictive Modeling with Heterogeneous Sources.

Analysis of Chernoff criterion for linear dimensionality reduction.

svdPPCS an effective singular value decomposition%2Dbased method for conserved and divergent co%2Dexpression gene module identification.

Visual data security and management for smart cities.


Heterogeneous cross domain ranking in latent space.


Relaxed Transfer of Different Classes via Spectral Partition.
Abstract. Most existing transfer learning techniques are limited to problems of knowledge transfer across tasks sharing the same set of class labels. In this paper, however, we relax this constraint and propose a spectral-based solution that aims at unveiling the intrinsic structure of the data and generating a partition of the target data, by transferring the eigenspace that well separates the source data. Furthermore, a clusteringbased KL divergence is proposed to automatically adjust how much to transfer. We evaluate the proposed model on text and image datasets where class categories of the source and target data are explicitly different, e.g., 3-classes transfer to 2-classes, and show that the proposed approach improves other baselines by an average of 10 % in accuracy. The source code and datasets are available from the authors. 1
Universal Learning over Related Distributions and Adaptive Graph Transduction.

A Thunderstorm Forecast Model Based on Weighted SVM and Data Field.

Mining Interesting Ratio Patterns over a Stream Sliding Window.

Cross domain distribution adaptation via kernel mapping.
When labeled examples are limited and difficult to obtain, transfer learning employs knowledge from a source domain to improve learning accuracy in the target domain. However, the assumption made by existing approaches, that the marginal and conditional probabilities are directly related between source and target domains, has limited applicability in either the original space or its linear transformations. To solve this problem, we propose an adaptive kernel approach that maps the marginal distribution of targetdomain and source-domain data into a common kernel space, and utilize a sample selection strategy to draw conditional probabilities between the two domains closer. We formally show that under the kernel-mapping space, the difference in distributions between the two domains is bounded; and the prediction error of the proposed approach can also be bounded. Experimental results demonstrate that the proposed method outperforms both traditional inductive classifiers and the state-of-the-art boosting-based transfer algorithms on most domains, including text categorization and web page ratings. In particular, it can achieve around 10 % higher accuracy than other approaches for the text categorization problem. The source code and datasets are available from the authors.
Heterogeneous source consensus learning via decision propagation and negotiation.
Nowadays, enormous amounts of data are continuously generated not only in massive scale, but also from different, sometimes conflicting, views. Therefore, it is important to consolidate different concepts for intelligent decision making. For example, to predict the research areas of some people, the best results are usually achieved by combining and consolidating predictions obtained from the publication network, co-authorship network and the textual content of their publications. Multiple supervised and unsupervised hypotheses can be drawn from these information sources, and negotiating their differences and consolidating decisions usually yields a much more accurate model due to the diversity and heterogeneity of these models. In this paper, we address the problem of “consensus learning ” among competing hypotheses, which either rely on outside knowledge (supervised learning) or internal structure (unsupervised clustering). We argue that consensus learning is an NP-hard problem and thus propose to solve it by an efficient heuristic method. We construct a belief graph to first propagate predictions from supervised models to the unsupervised, and then negotiate and reach consensus among them. Their final decision is further consolidated by calculating each model’s weight based on its degree of consistency with other models. Experiments are conducted on 20 Newsgroups data, Cora research papers, DBLP author-conference network, and Yahoo! Movies datasets, and the results show that the proposed method improves the classification accuracy and the clustering quality measure (NMI) over the best base model by up to 10%. Furthermore, it runs in time proportional to the number of instances, which is very efficient for large-scale data sets.
Latent space domain transfer between high dimensional overlapping distributions.
Transferring knowledge from one domain to another is challenging due to a number of reasons. Since both conditional and marginal distribution of the training data and test data are non-identical, model trained in one domain, when directly applied to a different domain, is usually low in accuracy. For many applications with large feature sets, such as text document, sequence data, medical data, image data of different resolutions, etc. two domains usually do not contain exactly the same features, thus introducing large numbers of “missing values”when considered over the union of features from both domains. In other words, its marginal distributions are at most overlapping. In the same time, these problems are usually high dimensional, such as, several thousands of features. Thus, the combination of high
Bagging.
Bagging predictors is a method for generating multiple versions of a predictor and using these to get an aggregated predictor. The aggregation averages over the versions when predicting a numerical outcome and does a plurality vote when predicting a class. The multiple versions are formed by making bootstrap replicates of the learning set and using these as new learning sets. Tests on real and simulated data sets using classification and regression trees and subset selection in linear regression show that bagging can give substantial gains in accuracy. The vital element is the instability of the prediction method. If perturbing the learning set can cause significant changes in the predictor constructed, then bagging can improve accuracy. 1. Introduction A learning set of L consists of data f(y n ; x n ), n = 1; : : : ; Ng where the y&#039;s are either class labels or a numerical response. We have a procedure for using this learning set to form a predictor &#039;(x; L) --- if the input is x we ...
Breaking the computational barrier a divide%2Dconquer and aggregate based approach for Alu insertion site characterisation.

A Fair Mobile Payment Protocol.

Actively Transfer Domain Knowledge.

Graph%2DBased Iterative Hybrid Feature Selection.

Direct mining of discriminative and essential frequent patterns via model%2Dbased search tree.

Knowledge transfer via multiple model local structure mapping.
The effectiveness of knowledge transfer using classification algorithms depends on the difference between the distribution that generates the training examples and the one from which test examples are to be drawn. The task can be especially difficult when the training examples are from one or several domains different from the test domain. In this paper, we propose a locally weighted ensemble framework to combine multiple models for transfer learning, where the weights are dynamically assigned according to a model’s predictive power on each test example. It can integrate the advantages of various learning algorithms and the labeled information from multiple training domains into one unified classification model, which can then be applied on a different domain. Importantly, different from many previously proposed methods, none of the base learning method is required to be specifically designed for transfer learning. We show the optimality of a locally weighted ensemble framework as a general approach to combine multiple models for domain transfer. We then propose an implementation of the local weight assignments by mapping the structures of a model onto the structures of the test domain, and then weighting each model locally according to its consistency with the neighborhood structure around the test example. Experimental results on text classification, spam filtering and intrusion detection data sets demonstrate significant improvements in classification accuracy gained by the framework. On a transfer learning task of newsgroup message categorization, the proposed locally weighted ensemble framework achieves 97 % accuracy when the best single model predicts correctly only on 73 % of the test examples. In summary, the improvement in accuracy is over 10 % and up to 30 % across different problems.
Forward Semi%2Dsupervised Feature Selection.

Forward Semi%2Dsupervised Feature Selection.

Clustering over Evolving Data Streams Based on Online Recent%2DBiased Approximation.

Type%2DIndependent Correction of Sample Selection Bias via Structural Discovery and Re%2Dbalancing.
Sample selection bias is a common problem in many real world applications, where training data are obtained under realistic constraints that make them follow a different distribution from the future testing data. For example, in the application of hospital clinical studies, it is common practice to build models from the eligible volunteers as the training data, and then apply the model to the entire populations. Because these volunteers are usually not selected at random, the training set may not be drawn from the same distribution as the test set. Thus, such a dataset suffers from “sample selection bias ” or “covariate shift”. In the past few years, much work has been proposed to reduce sample selection bias, mainly by statically matching the distribution between training set and test set. But in this paper, we do not explore the different distributions directly. Instead, we propose
Classifying Data Streams with Skewed Class Distributions and Concept Drifts.

Forecasting skewed biased stochastic ozone days analyses%2C solutions and beyond.

Forecasting skewed biased stochastic ozone days analyses%2C solutions and beyond.

Optimizing Supply Chains through Service%2DOriented Architecture.

Two Fair Payment Protocols for E%2DCommerce Transaction.

A Research on Construction of SOAs for Enterprises in China.

Image Hallucination Using Neighbor Embedding over Visual Primitive Manifolds.

On Appropriate Assumptions to Mine Data Streams Analysis and Practice.
Recent years have witnessed an increasing number of studies in stream mining, which aim at building an accurate model for continuously arriving data. Somehow most existing work makes the implicit assumption that the training data and the yet-to-come testing data are always sampled from the “same distribution”, and yet this “same distribution” evolves over time. We demonstrate that this may not be true, and one actually may never know either “how ” or “when ” the distribution changes. Thus, a model that fits well on the observed distribution can have unsatisfactory accuracy on the incoming data. Practically, one can just assume the bare minimum that learning from observed data is better than both random guessing and always predicting exactly the same class label. Importantly, we formally and
Multi%2Dagent Optimization Design for Multi%2Dresource Job Shop Scheduling Problems.

Semi%2Dsupervised Cast Indexing for Feature%2DLength Films.

A General Framework for Mining Concept%2DDrifting Data Streams with Skewed Distributions.
In recent years, there have been some interesting studies on predictive modeling in data streams. However, most such studies assume relatively balanced and stable data streams but cannot handle well rather skewed (e.g., few positives but lots of negatives) and stochastic distributions, which are typical in many data stream applications. In this paper, we propose a new approach to mine data streams by estimating reliable posterior probabilities using an ensemble of models to match the distribution over under-samples of negatives and repeated samples of positives. We formally show some interesting and important properties of the proposed framework, e.g., reliability of estimated probabilities on skewed positive class, accuracy of estimated probabilities, efficiency and scalability. Experiments are performed on several synthetic as well as real-world datasets with skewed distributions, and they demonstrate that our framework has substantial advantages over existing approaches in estimation reliability and predication accuracy. 1
On Sample Selection Bias and Its Efficient Correction via Model Averaging and Unlabeled Examples.

Challenges and Experience in Prototyping a Multi%2DModal Stream Analytic and Monitoring Application on System S.

FGF A web tool for Fishing Gene Family in a whole genome database.

An Traceable Electronic Cash Model Based on Group Signature.

Locally Linear Models on Face Appearance Manifolds with Application to Dual%2DSubspace Based Classification.

Face Recognition with Image Sets Using Hierarchically Extracted Exemplars from Appearance Manifolds.
An unsupervised nonparametric approach is proposed to automatically extract representative face samples (exemplars) from a video sequence or an image set for multipleshot face recognition. Motivated by a nonlinear dimensionality reduction algorithm called Isomap, we use local neighborhood information to approximate the geodesic distances between face images. A hierarchical agglomerative clustering (HAC) algorithm is then applied to group similar faces together based on the estimated geodesic distances which approximate their locations on the appearance manifold. We define the exemplars as cluster centers for template matching at the subsequent testing stage. The final recognition is the outcome of a majority voting scheme which combines the decisions from all the individual frames in the test set. Experimental results on a 40-subject video database demonstrate the effectiveness and flexibility of our proposed method. 
Discovering Unrevealed Properties of Probability Estimation Trees On Algorithm Selection and Performance Explanation.

Forecasting Skewed Biased Stochastic Ozone Days Analyses and Solutions.

Optimize Cooperative Agents with Organization in Distributed Scheduling System.

Facial Expression Recognition using Advanced Local Binary Patterns
This paper proposes a novel facial expression recognition approach based on two sets of features extracted from the face images: texture features and global appearance features. The first set is obtained by using the extended local binary patterns in both intensity and gradient maps and computing the Tsallis entropy of the Gabor filtered responses. The second set of features is obtained by performing nullspace based linear discriminant analysis on the training face images. The proposed method is evaluated by extensive experiments on the JAFFE database, and compared with two widely used facial expression recognition approaches. Experimental results show that the proposed approach maintains high recognition rate in a wide range of resolution levels and outperforms the other alternative methods. Index Terms — Face recognition, Machine vision, 1.
A general framework for accurate and fast regression by data summarization in random decision trees.

Reverse testing an efficient framework to select amongst classifiers under sample selection bias.
One of the most important assumptions made by many classification algorithms is that the training and test sets are drawn from the same distribution, i.e., the so-called “stationary distribution assumption” that the future and the past data sets are identical from a probabilistic standpoint. In many domains of real-world applications, such as marketing solicitation, fraud detection, drug testing, loan approval, sub-population surveys, school enrollment among others, this is rarely the case. This is because the only labeled sample available for training is biased in different ways due to a variety of practical reasons and limitations. In these circumstances, traditional methods to evaluate the expected generalization error of classification algorithms, such as structural risk minimization, ten-fold cross-validation, and leave-one-out validation, usually return poor estimates of which classification algorithm, when trained on biased dataset, will be the most accurate for future unbiased dataset, among a number of competing candidates. Sometimes, the estimated order of the learning algorithms ’ accuracy could be so poor that it is not even better than random guessing. Therefore, a method to determine the most accurate learner is needed for data mining under sample selection bias for many real-world applications. We present such an approach that can determine which learner will perform the best on an unbiased test set, given a possibly biased training set, in a fraction of the computational cost to use cross-validation based approaches.
When Efficient Model Averaging Out%2DPerforms Boosting and Bagging.
Abstract. Bayesian model averaging also known as the Bayes optimal classifier (BOC) is an ensemble technique used extensively in the statistics literature. However, compared to other ensemble techniques such as bagging and boosting, BOC is less known and rarely used in data mining. This is partly due to model averaging being perceived as being inefficient and because bagging and boosting consistently outperforms a single model, which raises the question: “Do we even need BOC in datamining?”. We show that the answer to this question is “yes ” by illustrating that several recent efficient model averaging approaches can significantly outperform bagging and boosting in realistic difficult situations such as extensive class label noise, sample selection bias and many-class problems. To our knowledge the insights that model averaging can outperform bagging and boosting in these situations has not been published in the machine learning, mining or statistical communities. 1
Video%2DBased Face Recognition Using Bayesian Inference Model.

Effective Estimation of Posterior Probabilities Explaining the Accuracy of Randomized Decision Tree Approaches.

An Improved Categorization of Classifier%27s Sensitivity on Sample Selection Bias.
A recent paper categorizes classifier learning algorithms according to their sensitivity to a common type of sample selection bias where the chance of an example being selected into the training sample depends on its feature vector   but not (directly) on its class label y. A classifier learner is categorized as &#034;local&#034; if it is insensitive to this type of sample selection bias, otherwise, it is considered &#034;global&#034;. In that paper, the true model is not clearly distinguished from the model that the algorithm outputs. In their discussion of Bayesian classifiers, logistic regression and hard-margin support vector machines, the true model (or the model that generates the true class label for every example) is implicitly assumed to be contained in the model space of the learner, and the true class probabilities and model estimated class probabilities are assumed to asymptotically converge as the training data set size increases. However, in the discussion of naive Bayes, decision trees and soft-margin support vector machines, the model space is assumed not to contain the true model, and these three classification algorithms are instead argued to be &#034;global learners&#034;. Here we argue that most classifier learning algorithms including those just discussed may or may not be affected by sample selection bias; this will depend on the dataset as well as the heuristics or inductive bias implied by the classifier learning algorithm and their appropriateness to the particular dataset. We make use of our earlier experimental results and produce additional results to illustrate our claims.
Local manifold matching for face recognition.

Maximizing Tree Diversity by Building Complete%2DRandom Decision Trees.

Making Data Mining Models Useful to Model Non%2Dpaying Customers of Exchange Carriers.

On the Optimality of Probability Estimation by Random Decision Trees.

Mining Extremely Skewed Trading Anomalies.

Combining Null Space%2Dbased Gabor Features for Face Recognition.
We propose a novel face recognition strategy combining various discriminating Gabor features in multi-scales and multi-orientations. A bank of well-chosen Gabor filters is applied on the image to construct a group of feature vectors, and then the Null Space-based LDA (NLDA) is performed simultaneously on each orientation channel and the original image to give 5 component classifier outputs, which are then combined to increase the final recognition rate. Experimental results on the FERET database demonstrate the effectiveness and flexibility of our proposed method. 1.
Decision Tree Evolution Using Limited Number of Labeled Data Items from Drifting Data Streams.
Most previously proposed mining methods on data streams make an unrealistic assumption that “labelled” data stream is readily available and can be mined at anytime. However, in most real-world problems, labelled data streams are rarely immediately available. Due to this reason, models are reconstructed only when labelled data become available periodically. This passive stream mining model has several drawbacks. We propose a new concept of demand-driven active data mining. In active mining, the loss of the model is either continuously guessed without using any true class labels or estimated, whenever necessary, from a small number of instances whose actual class labels are verified by paying an affordable cost. When the estimated loss is more than a tolerable threshold, the model evolves by using a small number of instances with verified true class labels. Previous work on active mining concentrates on error guess and estimation. In this paper, we discuss several approaches on decision tree evolution. 1
Combining Null Space%2Dbased Gabor Features for Face Recognition.
We propose a novel face recognition strategy combining various discriminating Gabor features in multi-scales and multi-orientations. A bank of well-chosen Gabor filters is applied on the image to construct a group of feature vectors, and then the Null Space-based LDA (NLDA) is performed simultaneously on each orientation channel and the original image to give 5 component classifier outputs, which are then combined to increase the final recognition rate. Experimental results on the FERET database demonstrate the effectiveness and flexibility of our proposed method. 1.
Systematic data selection to mine concept%2Ddrifting data streams.

Active Mining of Data Streams.
Most previously proposed mining methods on data streams make an unrealistic assumption that “labelled” data stream is readily available and can be mined at anytime. However, in most real-world problems, labelled data streams are rarely immediately available. Due to this reason, models are reconstructed only when labelled data become available periodically. This passive stream mining model has several drawbacks. We propose a new concept of demand-driven active data mining. In active mining, the loss of the model is either continuously guessed without using any true class labels or estimated, whenever necessary, from a small number of instances whose actual class labels are verified by paying an affordable cost. When the estimated loss is more than a tolerable threshold, the model evolves by using a small number of instances with verified true class labels. Previous work on active mining concentrates on error guess and estimation. In this paper, we discuss several approaches on decision tree evolution. 1
A Fast Algorithm for Subspace Clustering by Pattern Similarity.

StreamMiner A Classifier Ensemble%2Dbased Engine to Mine Concept%2Ddrifting Data Streams.
We demonstrate StreamMiner, a random decision-tree ensemble based engine to mine data streams. A fundamental challenge in data stream mining applications (e.g., credit card transaction authorization, security buysell transaction, and phone call records, etc) is concept-drift or the discrepancy between the previously learned model and the true model in the new data. The basic problem is the ability to judiciously select data and adapt the old model to accurately match the changed concept of the data stream. StreamMiner uses several techniques to support mining over data streams with possible concept-drifts. We demonstrate the following two key functionalities of StreamMiner: 1. Detecting possible concept-drift on the fly when the trained streaming model is used to classify incoming data streams without knowing the ground truth. 2. Systematic data selection of old data and new data chunks to compute the optimal model that best fits on the changing data streams.
Using artificial anomalies to detect unknown and known network intrusions.
Intrusion detection systems (IDSs) must be capable of detecting new and unknown attacks, or anomalies. We study the problem of building detection models for both pure anomaly detection and combined misuse and anomaly detection (i.e., detection of both known and unknown intrusions) . We propose an algorithm to generate artificial anomalies to coerce the inductive learner into discovering an accurate boundary between known classes (normal connections and known intrusions) and anomalies. Empirical studies show that our pure anomaly detection model trained using normal and artificial anomalies is capable of detecting more than 77% of all unknown intrusion classes with more than 50% accuracy per intrusion class. The combined misuse and anomaly detection models are as accurate as a pure misuse detection model in detecting known intrusions and are capable of detecting at least 50% of unknown intrusion classes with accuracy measurements between 75% and 100% per class.
Cross%2DFeature Analysis for Detecting Ad%2DHoc Routing Anomalies.

Indexing Weighted%2DSequences in Large Databases.
We present an index structure for managing weighted sequences in large databases. A weighted-sequence is defined as a two-dimensional structure where each element in the sequence is associated with a weight. A series of network events, for instance, is a weighted-sequence in that each event has a timestamp. Querying a large sequence database by events&#039; occurrence patterns is a first step towards understanding the temporal causal relationships among the events. The index structure proposed in this paper enables us to efficiently retrieve from the database all subsequences, possibly non-contiguous, that match a given query sequence both by events and by weights. The index method also takes into consideration the non-uniform frequency distribution of events in the sequence data. In addition, our method finds a broad range of applications in indexing scientific data consisting of multiple numerical columns for discovery of correlations among these columns. For instance, indexing a DNA micro-array that records expression levels of genes under different conditions enables us to search for genes whose responses to various experimental perturbations follow a given pattern. We demonstrate, using real-world data sets, that our method is effective and efficient.
Is random model better%3F On its accuracy and efficiency.

Inductive Learning in Less Than One Sequential Data Scan.

Mining concept%2Ddrifting data streams using ensemble classifiers.
Recently, mining data streams with concept drifts for actionable insights has become an important and challenging task for a wide range of applications including credit card fraud protection, target marketing, network intrusion detection, etc. Conventional knowledge discovery tools are facing two challenges, the overwhelming volume of the streaming data, and the concept drifts. In this paper, we propose a general framework for mining concept-drifting data streams using weighted ensemble classifiers. We train an ensemble of classification models, such as C4.5, RIPPER, naive Bayesian, etc., from sequential chunks of the data stream. The classifiers in the ensemble are judiciously weighted based on their expected classification accuracy on the test data under the time-evolving environment. Thus, the ensemble approach improves both the efficiency in learning the model and the accuracy in performing classification. Our empirical study shows that the proposed methods have substantial advantage over single-classifier approaches in prediction accuracy, and the ensemble framework is effective for a variety of classification models.
ViST A Dynamic Index Method for Querying XML Data by Tree Structures.
much research has been done in providing flexible query facilities to extract data from structured XML documents. In this paper, we propose ViST, a novel index structure for searching XML documents. By representing both XML documents and XML queries in structure-encoded sequences, we show that querying XML data is equivalent to finding subsequence matches. Unlike index methods that disassemble a query into multiple sub-queries, and then join the results of these sub-queries to provide the final answers, ViST uses tree structures as the basic unit of query to avoid expensive join operations. Furthermore, ViST provides a unified index on both content and structure of the XML documents, hence it has a performance advantage over methods indexing either just content or structure. ViST supports dynamic index update, and it relies solely on B    Trees without using any specialized data structures that are not well supported by DBMSs. Our experiments show that ViST is e#ective, scalable, and e#cient in supporting structural queries.
Pruning and Dynamic Scheduling of Cost%2DSensitive Ensembles.

An Index Structure for Pattern Similarity Searching in DNA Microarray Dat.

A Fully Distributed Framework for Cost%2DSensitive Data Mining.
In this paper, we propose a fully distributed system (as compared to centralized and partially distributed systems) for cost-sensitive data mining. Experimental results have shown that this approach achieves higher accuracy than both the centralized and partially distributed learning methods, however, it incurs much less training time, neither communication nor computation overhead. 1.
Progressive Modeling.
Presently, inductive learning is still performed in a frustrating batch process. The user has little interaction with the system and no control over the final accuracy and training time. If the accuracy of the produced model is too low, all the computing resources are misspent. In this paper, we propose a progressive modeling framework. In progressive modeling, the learning algorithm estimates online both the accuracy of the final model and remaining training time. If the estimated accuracy is far below expectation, the user can terminate training prior to completion without wasting further resources. If the user chooses to complete the learning process, progressive modeling will compute a model with expected accuracy in expected time. We describe one implementation of progressive modeling using ensemble of classifiers.
Empirical Comparison of Various Reinforcement Learning Strategies for Sequential Targeted Marketing.
We empirically evaluate the performance of various reinforcement learning methods in applications to sequential targeted marketing. In particular, we propose and evaluate a progression of reinforcement learning methods, ranging from the &#034;direct&#034; or &#034;batch&#034; methods to &#034;indirect&#034; or &#034;simulation based&#034; methods, and those that we call &#034;semidirect &#034; methods that fall between them. We conduct a number of controlled experiments to evaluate the performance of these competing methods. Our results indicate that while the indirect methods can perform better in a situation in which nearly perfect modeling is possible, under the more realistic situations in which the system&#039;s modeling parameters have restricted attention, the indirect methods&#039; performance tend to degrade. We also show that semi-direct methods are effective in reducing the amount of computation necessary to attain a given level of performance, and often result in more profitable policies.
A Framework for Scalable Cost%2Dsensitive Learning Based on Combing Probabilities and Benefits.

Ensemble%2Dbased Adaptive Intrusion Detection.

Toward Cost%2DSensitive Modeling for Intrusion Detection and Response.
Intrusion detection systems (IDSs) must maximize the realization of security goals while minimizing costs. In this paper, we study the problem of building cost-sensitive intrusion detection models. We examine the major cost factors associated with an IDS, which include development cost, operational cost, damage cost due to successful intrusions, and the cost of manual and automated response to intrusions. These cost factors can be qualified according to a defined attack taxonomy and site-specific security policies and priorities. We define cost models to formulate the total expected cost of an IDS, and present cost-sensitive machine learning techniques that can produce detection models that are optimized for user-defined cost metrics. Empirical experiments show that our cost-sensitive modeling and deployment techniques are effective in reducing the overall cost of intrusion detection. 1
Using Artificial Anomalies to Detect Unknown and Known Network Intrusions.
Intrusion detection systems (IDSs) must be capable of detecting new and unknown attacks, or anomalies. We study the problem of building detection models for both pure anomaly detection and combined misuse and anomaly detection (i.e., detection of both known and unknown intrusions) . We propose an algorithm to generate artificial anomalies to coerce the inductive learner into discovering an accurate boundary between known classes (normal connections and known intrusions) and anomalies. Empirical studies show that our pure anomaly detection model trained using normal and artificial anomalies is capable of detecting more than 77% of all unknown intrusion classes with more than 50% accuracy per intrusion class. The combined misuse and anomaly detection models are as accurate as a pure misuse detection model in detecting known intrusions and are capable of detecting at least 50% of unknown intrusion classes with accuracy measurements between 75% and 100% per class.
Mining System Audit Data Opportunities and Challenges.
Intrusion detection is an essential component of computer security mechanisms. It requires accurate and efficient analysis of a large amount of system and network audit data. It can thus be an application area of data mining. There are several characteris-tics of audit data: abundant raw data, rich system and network semantics, and ever &amp;quot;streaming&amp;quot;. Ac-cordingly, when developing data mining approaches, we need to focus on: feature extraction and con-struction, customization of (general) algorithms ac-cording to semantic information, and optimization of execution efficiency of the output models. In this paper, we describe a data mining framework for min-ing audit data for intrusion detection models. We discuss its advantages and limitations, and outline the open research problems. 1
Data Mining%2Dbased Intrusion Detectors An Overview of the Columbia IDS Project.

A Multiple Model Cost%2DSensitive Approach for Intrusion Detection.
Intrusion detection systems (IDSs) need to maximize security while minimizing costs. In this paper, we study the problem of building cost-sensitive intrusion detection models to be used for realtime detection. We briefly discuss the major cost factors in IDS, including consequential and operational costs. We propose a multiple model cost-sensitive machine learning technique to produce models that are optimized for user-defined cost metrics. Empirical experiments in off-line analysis show a reduction of approximately 97% in operational cost over a single model approach, and a reduction of approximately 30% in consequential cost over a pure accuracy-based approach.
Web%2Dcollaborative filtering recommending music by crawling the Web.

AdaCost Misclassification Cost%2DSensitive Boosting.
AdaCost, a variant of AdaBoost, is a misclassification cost-sensitive boosting method. It uses the cost of misclassifications to update the training distribution on successive boosting rounds. The purpose is to reduce the cumulative misclassification cost more than AdaBoost. We formally show that AdaCost reduces the upper bound of cumulative misclassification cost of the training set. Empirical evaluations have shown significant reduction in the cumulative misclassification cost over AdaBoost without consuming additional computing power. 1
The Application of AdaBoost for Distributed%2C Scalable and On%2DLine Learning.
1 Introduction Learning from very large and distributed databases imposes major performance challenges for data mining. Many databases have grown too large to fit into main memory. Learning a monolithic classifier will be very slow since it requires all data be held in main memory. Some databases are inherently distributed and cannot be brought into a single site for a variety of reasons. It is very hard to learn a single classifier under these constraints. One such application is to learn a fraud detection model from different participating credit card companies whose datasets are huge and strictly distributed. Yet there are some databases that have large amounts of new data available periodically or in real time. To re-learn a new model of the whole data set each time an increment is provided is both expensive and inefficient.
Learning Page%2DIndependent Heuristics for Extracting Data from Web Pages.
One bottleneck in implementing a system that intelligently queries the Web is developing &#034;wrappers&#034;---programs that extract data from Web pages. Here we describe a method for learning general, page-independent heuristics for extracting data from HTML documents. The input to our learning system is a set of working wrapper programs, paired with HTML pages they correctly wrap. The output is a general procedure for extracting data that works for many formats and many pages. In experiments with a collection of 84 constrained but realistic extraction problems, we demonstrate that 30% of the problems can be handled perfectly by learned extraction heuristics, and around 50% can be handled acceptably. We also demonstrate that learned page-independent extraction heuristics can substantially improve the performance of methods for learning page-specific wrappers. Keywords: information integration, machine learning, extraction. 1 Introduction  A number of recent systems operate by taking informatio...
